Stereoscopic camera and system

ABSTRACT

The present invention provides a stereoscopic camera containing a binocular instrument and an integrated Stereoscopic Image Acquisition Device (SIAD), and improved methods for acquiring stereoscopic image data. The invention also provides master-slave control of adjustable channels in a stereoscopic camera. The camera can be free standing and may contain integrated power supplies, image processing units, storage mechanisms, or display mechanisms. The invention circumvents the need for, and limitations of, binocular eyepieces, and is capable of producing high resolution, real-time image data while avoiding or mitigating the deleterious effects of spurious parallax, IPD, and convergence.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 11/256,497 filed on Oct. 21, 2005 and claims the benefit ofU.S. Provisional Patent Application No. 60/833,117 filed on Jul. 24,2006, the entire disclosures of which are incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention generally relates to methods for stereoscopic imagingand, more specifically, to a device that acquires stereoscopic images.

2. Description of Related Art

The use of stereoscopy, in which the user sees left- and right-eye viewsand forms a three dimensional image through stereopsis, is common inmany areas of science, medicine and entertainment. The use of abinocular to provide a magnified stereoscopic image of distant objectsto a user's eyes is also common. Binoculars are used for observation,surveillance, and many other purposes.

The optical image generated by a binocular is typically viewed througheyepieces. However, the use of eyepieces in binocular systems is oftenproblematic. Furthermore, only one observer at a time can view imagesgenerated by the binocular and the observer can no longer see what ishappening in the surrounding environment. In addition, a binocular, assuch, cannot store images or sequences of images for later playback,process them in special ways, or transmit them to remote viewing sites.There are also situations in which it is desirable to remotely view orrecord a stereoscopic image of a location without involving a person totake the image.

Therefore, it is often desirable to use electronic imaging to acquireimages of a location, either for direct, real-time observing or forrecording Electronic imaging is a preferred method of the televisionbroadcasting, video and movie industries as well. The use of cameras andelectronic displays to acquire images is well known in the art,including the use of two cameras and a 3D display to give a stereoscopicimage.

However the two-camera systems have many disadvantages. Obtaining andmaintaining stereoscopic alignment (necessary for comfortable, long-termviewing) can be very difficult when two independent cameras are mountedon or comprise a binocular. The cameras generally protrude from thegeneral body of the device and are often mounted in a way that isfragile and prone to breakage. Protruding cameras can interfere withother apparatuses in the workspace, limiting possible usageconfigurations. The two-camera systems have generally double thebinocular and camera knobs and controls, resulting in an unwieldy devicedifficult to operate by a single user. Dual camera systems generallyrequire numerous mounting parts, resulting in less reliability and morecost than a single, integrated camera.

There are also problems with mounting and connecting the cameras todisplays or storage media. The use of two cameras requires multiplecables and connectors, resulting in less reliability and more difficultinstallation than a single cable/connector arrangement of the presentinvention. The two-camera system also typically requires two cameracontrol units (CCUs) and two storage devices, and requires that they besynchronized for best image quality. This significantly increases thecost of the system.

In addition, such cameras do not allow precise positioning of theimaging sensors to each other for best stereopsis and comfortableviewing, particularly when two off-the-shelf cameras are used. Cameraswhich are wide cannot be easily positioned side-by-side with closespacing. The cameras must be individually focused after mounting, and,should adjustments such as brightness and contrast be needed, eachcamera must be controlled individually. Where the cameras containirises, they must also be individually adjusted for each camera,resulting in the potential for unequal amounts of light entering eachcamera, which can lead to difficult 3D viewing and eyestrain. All thesefactors indicate that using such a system requires skill and can be verytime-consuming.

Image processing is also problematic in such systems. The cameras mustbe electronically linked in some way so that the two image streams aresynchronized, creating additional cost and complexity. Even if the datastreams are synchronized, generally the shutters are not perfectlysynchronized such that the nth pixel from one view was not captured atthe same time as the nth pixel from the other view, causing movingobjects to show spurious parallax when displayed. Furthermore, theimages acquired by the two cameras are generally taken directly to the3D display device. Therefore, should the user require image processing,storage, transmission, or display on alternative displays, additionalprocessing units are required, creating additional cost and complexity.

The cameras used in such two-camera systems also usually conform to theNTSC or PAL video standard, both of which suffer from low resolution,poor color fidelity, and motion artifacts (due to the interlaced natureof the raster scan). Recording and editing recorded content is alsoproblematic with the two-camera system. Recorders don't generally startand end synchronously, so the two tapes or files must somehow besynchronized, resulting in additional effort and expense. Editing mayneed to be performed twice—once to each file or tape.

Information relevant to attempts to address these problems can be foundin U.S. Pat. Nos. 4,418,993; 4,583,117; 4,879,596; 4,881,122; 5,164,827;5,438,386; 6,157,337, and 6,512,892.

However, each one of these references suffers from one or more of thefollowing disadvantages: the device or system creates two independentoutput signals; is not compact; does not provide sufficient imageprocessing, recording, or transmission capability; does not haveadequate resolution in real-time for many applications; is cumbersome oris not easily operated by a single user; is large and expensive; morethan one operator is generally needed to properly control all of therequired functions in order to provide good images,; it is difficult tosynchronize two separate cameras to the pixel level; two recordingdevices or image-capturing paths are required, resulting in additionalcomplexity and cost in acquiring and recording the images and editingthem as is often desirable; accessory image/data recording systems havea required start-up time prior to recording; uses significant power,requiring large batteries for mobile applications and emittingsignificant heat that could disturb sensitive environments; is morefragile than a single camera; or does not perform well if either or bothof the cameras uses automatic focusing, automatic exposure control orimage stabilization control, because such systems or devices heretoforehave not been synchronized for the two views from the two cameras;

Therefore, the use of binocular systems containing electronic cameras,recording devices and display therefore solves some of the eyepieceproblems but creates new ones, essentially making them impractical forroutine use.

SUMMARY OF THE INVENTION

The present invention provides improved devices and methods for viewingand recording images and, in particular, stereoscopic binocular images.

The invention relates to a compact stereoscopic camera capable ofproviding visual observation and recording of images of a location. Inparticular, the present invention provides a binocular instrument havingan integrated Stereoscopic Image Acquisition Device (SIAD) whichcircumvents the need for, and limitations of binocular eyepieces. Thecamera acquires and transfers high-resolution, real-time stereoscopicimage data in a single data stream, from stereoscopic still or movingviews of a location synchronized to the pixel level, to imageprocessing, recording, or display systems which may be included in anintegrated handheld device. The device performs the desired functionswithout protruding elements, numerous cables and connectors, and otheradditional components, and could be readily operated by a single user.

One aspect of the invention is a stereoscopic camera having a binocularinstrument and a stereoscopic image acquisition device. In oneembodiment, the camera contains mechanisms or structures designed toavoid spurious parallax. In another embodiment, the camera containsmechanisms or structures designed to control the effect of varying IPD.In yet another embodiment, the camera contains mechanisms or structuresdesigned to control the effects of varying convergence. In a furtherembodiment, the camera can provide a non-reflected view or desirableorientation of the location. In another embodiment, the camera containsmaster-slave control of adjustable channels. In yet another embodiment,the camera may be free standing and contains an integrated power source,image processing unit, and storage device. In yet another embodiment, adisplay mechanism is integrated into the camera.

A second aspect of the invention is a stereoscopic camera includingmaster-slave control of adjustable camera channels such that aspects ofthe views from channels are equalized, providing optimal stereopsis.

A third aspect of the invention is a method for acquiring stereoscopicimages of a location, the method including steps for interactivelyaligning vergence without producing substantially abrupt transitions inthe views used to acquire the stereoscopic images. In one embodiment,alignment could be achieved by a single user. In another embodiment,alignment could be achieved simultaneously with other cameraadjustments. In yet another embodiment, the method includes processesfor maintaining vertical position equalization in order to preventspurious parallax between the respective views.

A fourth aspect of the invention is a stereoscopic camera in which thefunctional elements of a binocular instrument and a stereoscopic imageacquisition device are integrated into a single package.

A fifth aspect of the invention is an interactive method for mitigatingthe effects of camera shake while acquiring stereoscopic images.

These and other further features and advantages of the invention will beapparent to those skilled in the art from the following detaileddescription, taken together with the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of one embodiment of the invention having astereoscopic camera system, including the system components and aviewer;

FIG. 2 is an expanded perspective view of the stereoscopic cameracomponents of the embodiment shown in FIG. 1; and

FIG. 3 is a perspective view of one embodiment of the inventioncontaining a deflecting element in addition to stereoscopic camerasystem components and a viewer.

DETAILED DESCRIPTION OF THE INVENTION

The present invention provides improved devices and methods for viewingand recording images and, in particular, binocular stereoscopic images.

Briefly, and in general terms, the present invention is directed to anoptical instrument having an integrated SIAD device. In particular, thepresent invention provides a binocular having an integrated SIAD devicewhich circumvents the need for, and limitations of, binocular eyepieces,and additionally includes numerous features not hitherto associated withbinoculars.

In particular, the present invention relates to a compact stereoscopiccamera capable of providing visual observation of a location. Inparticular, the present invention provides a binocular instrument havingan integrated Stereoscopic Image Acquisition Device (SIAD) whichcircumvents the need for, and limitations of, binocular eyepieces. Thecamera acquires and transfers high-resolution, real-time stereoscopicimage data in a single data stream, from stereoscopic still or movingviews of a location synchronized to the pixel level, to imageprocessing, recording, or display systems which may be included in anintegrated handheld device. The device performs the desired functionswithout protruding elements, numerous cables and connectors, and otheradditional components, and could be readily operated by a single user.

The following description presents preferred embodiments of theinvention representing the best mode contemplated for practicing theinvention. This description is not to be taken in a limiting sense butis made merely for the purpose of describing the general principles ofthe invention whose scope is defined by the appended claims.

Before addressing details of embodiments described below, some terms aredefined or clarified. As used herein, the terms “comprises,”“comprising,” “includes,” “including,” “has,” “having” or any othervariation thereof, are intended to cover a non-exclusive inclusion. Forexample, a process, method, article, or apparatus that comprises a listof elements is not necessarily limited to only those elements but mayinclude other elements not expressly listed or inherent to such process,method, article, or apparatus. Further, unless expressly stated to thecontrary, “or” refers to an inclusive or and not to an exclusive or. Forexample, a condition A or B is satisfied by any one of the following: Ais true (or present) and B is false (or not present), A is false (or notpresent) and B is true (or present), and both A and B are true (orpresent).

Also, use of the “a” or “an” are employed to describe elements andcomponents of the invention. This is done merely for convenience and togive a general sense of the invention. This description should be readto include one or at least one and the singular also includes the pluralunless it is obvious that it is meant otherwise.

Unless otherwise defined, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which this invention belongs. Furthermore, any definitionsused refer to the particular embodiments described herein and are not tobe taken as limiting; the invention includes equivalents for otherundescribed embodiments. Although methods and materials similar orequivalent to those described herein can be used in the practice ortesting of the present invention, suitable methods and materials aredescribed below. All publications, patent applications, patents, andother references mentioned herein are incorporated by reference in theirentirety. In case of conflict, the present specification, includingdefinitions, will control. In addition, the materials, methods, andexamples are illustrative only and not intended to be limiting.

As used herein, the term “beam” is intended to mean a rigid member orstructure supported at one or both ends, subject to bending forces froma direction perpendicular to its length. A beam can be made flexible ina direction and rigid in others.

As used herein, the term “binocular” is intended to mean an opticalinstrument consisting of two optical paths, each comprising one or moreoptical components, such as a lens, or combination thereof for focusinga stereoscopic image of a location or object therein on, for example,the eyes of a viewer or on a sensor. A binocular can be used formagnifying a small distant object but can also be used to provide ade-magnified view of a location, for example a wide-angle stereoscopicimage of a landscape location. By comparison, a microscope generally isused for magnification of small objects which are close and heldattached to a stationary portion of the microscope to which the opticalpath is focused. A binocular is generally used to observe objects atmore varying, farther and random distances.

As used herein, the term “camera” is intended to mean a device thatconsists of one or more lightproof chambers with one or more aperturesfitted with one or more in combination lens or other optical componentthrough which the image of an object is projected and focused onto asurface for recording (as on film, for example) or for translation intoelectrical impulses or data, for display or recording (as for televisionbroadcast, for example).

As used herein, the term “camera shake” is intended to mean the effectof unintended vibration and random motion imparted to a camera by theunsteadiness of the holding device, which is generally a user's hands, avehicle's mounting bracket or an unsteady base. Camera shake appears ona display as a bouncing or vibrating view.

As used herein, the term “centration” is intended to mean the accuracywith which the optical axis of a lens in an optical instrument coincideswith the mechanical axis of a mounting in the instrument for that lens.Poor centration can cause spurious parallax when optical components aremoved relative to one another or to a sensor.

As used herein the term “channel” when referring to a stereoscopiccamera is intended to mean the components required to acquire a view ofa stereoscopic image. One nonlimiting example of a channel consists of alens, optical path and one or more sensors to acquire the view as data.Channels are typically arranged such that the optical axes are coplanarand may converge at a distant point, and the input optical componentsare generally side by side.

As used herein the term “channel spacing” or “spacing of channels”refers to the distance between the optical paths at the inputs to theinput optical components of channels, often objective lenses. Channelspacing may be adjusted in order to change one or more of the views of alocation, thereby providing a desired perspective of the stereoscopicimage formed by the views.

As used herein, the term “controller” is intended to mean the componentof a system that contains the circuitry necessary to interpret andexecute instructions fed into the system. For example an acquisitionsystem may contain an acquisition controller. Representative graphicscontrollers include without limitation a graphics card, video card,video board, video display board, display adapter, video adapter,graphics adapter, image processing unit or combination thereof.

As used herein, the term “de-reflection” refers to reversing thereflecting effect of a deflecting element. If a mirror or otherdeflecting element is used between the object and a camera, for exampleto look around a corner, the view can be reversed electronically byreassigning the location of pixels within the views such that the viewpresented on the display is oriented as if the object was vieweddirectly.

As used herein, the term “electronic mechanisms for adjusting the sizeof an object” include automatic zoom and digital zoom.

As used herein, the term “equalize” or “equalized” is intended to meanto cause to correspond, or be like, in amount or degree as compared,including without limitation to make equal or to compensate fordifferences.

As used herein, the term “equalization of size” refers to having anobject appear at the appropriate size to each eye of the viewer.Generally for objects in front of a viewer an object will appear thesame size in each eye. As such, in a stereoscopic image it is importantthat an object appear the same size in each view in order for the viewerto form the best stereopsis. If the object does not appear to be thesame size, the size can be equalized by making the size of one viewequal to that of the other view of the object to correct the image.

As used herein, the term “equalization of vertical position”, verticalreferring to the direction perpendicular to the plane of the opticalaxes of two channels of a stereoscopic camera, refers to making anobject appear at the appropriate vertical position to each eye of theviewer. Generally a point on an object is at the same vertical positionin each view to avoid spurious parallax which detriments the viewer'sstereopsis. If the object does not appear to be at the same verticalposition, the position can be equalized by making the position of oneview equal to that of the other view of the object to correct the image.

As used herein, the term “free-standing” is intended to describe adevice which is sufficiently complete in construction such that noadditional devices are required for its operation. For example withoutlimitation, a camcorder can be free-standing as it runs on batteries andhas a built-in recording system; the user need have no other device tooperate it.

As used herein, the term “high-resolution” when referring tostereoscopic images is intended to mean at least about 1280 by 720pixels for each left or right view. It is contemplated that resolutionsof three times and eight times this minimum resolution may beimplemented depending on the state of technology for sensors anddisplays and depending on what cost is acceptable. On the other hand,the devices of the present invention may be implemented withoutlimitation with higher or lower resolutions for either one or both ofthe views.

As used herein, the term “image data” is intended to mean data producedby regular array detectors such as CMOS, CCDs and infrared devices. Thedata structures are created by the data acquisition systems oracquisition controller for use by observers, data reduction systems, andarchives.

As used herein the term “kinematic relationship” is intended to mean theability to deduce the motion of points on a device from the knowledge ofthe motion of other points on the device and the geometry of the device.For example without limitation, if one end of a long lever is lifted,say, 2 inches, it can be deduced from the kinematic relationship thatthe motion of the midpoint of the lever is about 1 inch.

As used herein, the term “lens” is intended to refer to a piece oftransparent material (such as, for example, glass) that has two oppositeregular surfaces, either both curved or one curved and the other plane,and that is used either singly or combined in an optical instrument forforming an image by focusing rays of light. “Lens” may refer to anindividual lens or a plurality of individual lenses acting incombination.

As used herein, the term “location” is intended to mean a position orsite marked by some distinguishing feature, including without limitationa place, scene, or point therein.

As used herein, the term “magnification” is intended to mean the ratioof the size of an image to the size of an object. It can be a relativeterm because an electronic image of an object imaged with a camera couldbe displayed on a large or a small display device and hence havedifferent magnifications resulting from identical image data.

As used herein, the term “mechanism” is intended to mean a process,technique, device or system for achieving a result. A mechanism may becontrolled in a variety of ways, including without limitationmechanically, electromechanically, electrically, or electronicallyoperated mechanisms.

As used herein, the term “optical” is intended to mean of or relating toor involving light or optics, including without limitation the use ofvisible radiation or combinations of visible and non-visible radiationto visualize objects.

As used herein, the term “optical component” is intended to mean a partof an optical system which deflects, refracts, restricts, focuses,manipulates, mirrors, modifies, filters or has some other intendedeffect on a beam of light including without limitation lenses, prisms,mirrors, and beamsplitters.

As used herein, the term “optical instrument” is intended to mean anyoptical instrument capable of generating images including withoutlimitation microscopes, endoscopes, binoculars, and telescopes.

As used herein, the term “optical path” is intended to mean thegenerally central ray in an optical system. Should the system have nocentral ray then the optical path is the general centerline of theaverage of all the rays.

As used herein, the term “optical device” is intended to mean any deviceor instrument capable of generating, sensing, capturing, processing,formatting, or storing images or image data.

As used herein, the term “optical instrument” is intended to mean anyoptical instrument capable of generating visible stereoscopic imagesincluding without limitation microscopes, endoscopes, binoculars, andtelescopes.

As used herein, the phrase “real time” is intended to mean that theimage data is acquired, processed, transmitted, or displayed at asufficiently high data rate and at a sufficiently low delay that objectson a display move smoothly, for example without user-noticeable judderor latency between object motion and display motion. Typically, thisoccurs when new images are acquired, processed, and transmitted at arate of at least about 24 frames per second (fps) and displayed at arate of at least about 45 fps and when the combined processing of thesystem has no more than about 1/30^(th) sec of delay.

As used herein, the phrase “single data stream” is intended to mean acombination of more than one individual data streams into a singlestream such that a desirable aspect of the data is maintained, such astiming or scale.

As used herein, the term “sensor” is intended to mean a device thatresponds to a stimulus, such as heat, light, or pressure, and generatesone or more signals that can be measured or interpreted.

As used herein, the term “shutter” is intended to mean a cameracomponent that allows light to enter by opening and closing an aperture.

As used herein, the term “spurious parallax” is intended to mean theparallax between views in a stereoscopic image which appears imperfectto the viewer. Vertical parallax of even a small amount results in poorstereopsis. For example, spurious parallax can be caused by non-planaroptical axes of channels, unequal magnification in channels, vibrationof the camera, distorted or unequal optical paths of channels andsimilar imperfections.

As used herein, the term “stereoscopic image” is intended to mean asingle image consisting of at least two views, one corresponding to aleft-eye view, i.e. the left view, and one corresponding to a right-eyeview, the right view.

As used herein, the term “stereoscopic image acquisition device” isintended to mean a device capable of acquiring stereoscopic images froman optical instrument, such as binoculars or the imaging componentsthereof. The device acquires and transfers high-resolution, real-timeimage data from stereoscopic still or moving images to image processing,recording, or display systems. The device performs the desired functionswithout protruding elements, numerous cables and connectors, and otheradditional components such as eyepieces, and can be readily adapted foruse with a variety of optical instruments.

As used herein, the term “telescope” is intended to mean an instrumentdesigned for the observation of remote objects, typically comprising aninput optical component at the distal end of a tube and one or moreoptical components at the proximal end, such tube providing a path forlight beams and possibly a path to change the spacing of opticalcomponents and hence focus or magnification, including withoutlimitation a tubular optical instrument for viewing distant objects bymeans of the refraction of light rays through a lens or the reflectionof light rays by a concave mirror.

As used herein, the term “vergence” is intended to mean the ability ofthe optical axes of the eyes or of an optical instrument to rotatetoward or away from each other to remain pointed at an object as itapproaches or moves away.

As used herein, the term “view” is intended to mean extent or range ofvision.

Attention is now directed to more specific details of embodiments thatillustrate but not limit the invention.

General Description

One embodiment of the invention provides a stereoscopic camera thatincludes a Stereoscopic Image Acquisition Device (SIAD) having anacquisition controller and a binocular optical instrument that may beattached to or built into the SIAD.

FIGS. 1-3 illustrate three embodiments of the invention having a varietyof components and applications. FIG. 1 illustrates one embodiment of theStereoscopic Camera and System and its components, and a viewer. Thecomponents are labeled as outlined below:

-   -   1. Object in a location being imaged;    -   2. Optical axes;    -   3. User interface, in this embodiment a pistol-grip with        controls;    -   4. Stereoscopic Image Acquisition Device (SIAD);    -   5. Optical instrument, in this embodiment it is a binocular        having two telescopes that is attached to the SIAD;    -   6. Stereoscopic display, in this embodiment an autostereoscopic        flat panel;    -   7. IPU, in this embodiment a display controller is included in        the IPU;    -   8. Viewer, in approximate viewing position in this embodiment to        see both object and 3D image; and    -   9. Image of object in location, appearing in 3D to the viewer.

FIG. 2 shows a close-up of several components of the embodiment shown inFIG. 1 which are labeled as outlined below:

-   -   2. Optical axes;    -   3. User interface, in this embodiment it is a pistol grip with        trigger for image capture and thumb-operated joystick for other        input functions;    -   4. Stereoscopic Image Acquisition Device. In this embodiment the        connections (not shown) from sensors to acquisition controller        are flexible circuits, allowing mutual movement between sensors        and acquisition controller;    -   5. Optical instrument, in this embodiment is a binocular having        two telescopes, (5 a) and (5 b) respectively;    -   6. Stereoscopic display, in this embodiment for images and user        interface;    -   7. IPU; in this embodiment a display controller is included in        the IPU;    -   10 a and 10 b. Means to adjust the convergence of the optical        axes, in this embodiment it is a flexing beam (10 a) with a        manually-driven screw (10 b);    -   11 a and 11 b. Means to adjust the interpupillary distance        (IPD), in this embodiment it is a slider traveling on a rail (11        a), controlled by an electromechanical actuator (11 b);    -   12. Means to adjust the vertical alignment of the cameras'        images, in this embodiment it is an actuator-driven screw; and    -   13. Battery pack.

Components

In one embodiment, the invention provides a stereoscopic camera systemthat includes a Stereoscopic Image Acquisition Device 4 (SIAD), abinocular optical instrument 5 that may be attached to or built into theSIAD, and a display mechanisms for displaying stereoscopic images 6generated by the optical instrument and SIAD. In yet another embodiment,the system may include an image-processing unit 7 (IPU). In anotherembodiment, the system may include a battery or other power source 13 toprovide power to the system. Further embodiments may contain no SIAD butmay contain other components to perform similar functions.

The image processing unit 7 as well as the display 6 and a battery 13(to provide power) may be attached to the device and integrated into ahousing, resulting in a complete, integrated, one-piece device thatprovides all the necessary functionality including: mechanisms for (1)forming a stereoscopic electronic image of an object or location, (2)processing such image data and (3) displaying a magnified, unmagnified,or demagnified stereoscopic image of the object or location, in adesired orientation and condition (e.g. not inverted or reflected, or inany orientation desired by the user), in a convenient position on thestereoscopic display for the viewer, in real time, and in a device whichcould be portable.

In other embodiments, some or all of the functions of the IPU can bebuilt into circuitry, firmware and software inside the SIAD or elsewherein the system, such that a separate IPU component may not be required,possibly reducing the size and cost of the system. In yet otherembodiments, the one-piece device could be handheld in its use. Infurther embodiments, the display could be mounted separately with atethering data cable or wireless link to the SIAD or IPU.

In one embodiment, the display could be mounted to the chest of theuser, facing the user's view, such that his hands can be free to steerand operate the device or perform other tasks while observing the imageon the display. An additional advantage of such a system is that theuser can see the displayed image at the same time as his peripheralvision allows him to see the rest of the location or surroundings, orvice-versa. Additionally, the system could have the display mounteddirectly on a handheld device that may be the SIAD, to accomplish asimilar result. In these or other embodiments the display, or one ormore additional displays connected via a “splitter” device, could bemounted such that multiple viewers could see the displayed image.

Image Processing

Image processing could be performed on the data to reduce the perceivedeffect of camera shake on the viewed stereoscopic image. Time sequentialdata from both left and right binocular channels in combination could beused to calculate corrections of the data to negate the shake effect ofboth channels, providing image stabilization for the entire stereoscopicimage for example, or to otherwise cause a desired effect. Because theelectronic corrections can be calculated knowing the kinematicrelationship between the optical axes and because the data from thesensors is synchronized with each other the proper corrections can beapplied to the stereoscopic image. This is advantageous as compared tothe prior art, which applied the corrections to the views separately andasynchronously, resulting in spurious parallax of the stereoscopicimage. Alternatively the corrections could be applied to one or moreactuation mechanisms to alter the optical path or paths, such that theimage is corrected when it arrives at the sensor.

Data from both left and right channels could be used to calculatecorrections to the sensor exposure parameters or corrections to the dataitself to optimize the stereoscopic image and to balance the leftchannel with the right, providing simultaneous exposure or gain controlfor example. One embodiment of this could involve a simultaneousbaseline setting of the two channels to give equal imagecharacteristics, for example performing white balance simultaneously.

Image processing to compress or encode the data could be done to thesingle data stream or to one or both streams prior to their combination.

Displays

In these and other embodiments of the current invention, thestereoscopic display could be of any type as described in the relatedU.S. Utility Patent Application Ser. No. 60/762,577 filed on Jan. 29,2007 that can be further applicable to this invention. In embodimentswhere the user looks slightly downward to see the display but looks upover the display to see the location being imaged, and where the displayis of a type requiring the user to wear spectacles to see the imagestereoscopically, it could be advantageous for the spectacles to beconstructed like “reading glasses” whereby looking over the activeportion of such spectacles the user has an unobstructed view of thelocation. Alternatively, left-right stick-on films made from polarizers,retarders or other materials required for stereoscopic viewing of the 3Ddisplay, for viewers wearing other glasses, or polarized sunglasses,could be used. A graphic on the film could indicate proper orientation.A permanent or non-permanent adhering method could be used.

Another embodiment may have a display that could be switched fromdisplaying either left or right or both views. Another embodiment mayhave a display that presents either or both views together and has noprovision for stereopsis.

Storage

In other embodiments one or more Hard Disk Drives (HDD), Digital VideoDisks (DVD), solid-state or other similar data storage devices (SD),could be included in the IPU or elsewhere for storage, furtherprocessing and playback of images. Because the SIAD inputs, processesand displays images in generally real-time (as observed by a user), theSD interface circuitry could be constructed such that images can bestored essentially instantaneously, with no perceived latency after astart trigger has been activated. This has previously been a problemwith digital cameras, whereby a shutter button is pressed but the imageis not taken until after a noticeable delay. In some embodiments, thefile structure' may be split into parallel channels such that thebandwidth of data is split to allow that flowing to each SD to be withinthe SD's transfer rate. In other embodiments, the storage system couldmake the actual recording of the images on the SD with a delay from whenthey are captured in system volatile memory. In yet other embodiments,streaming full-motion stereoscopic image data continuously through theSD system, such that there exists some quantity of image data previouslystored, upon the occurrence of an event desirable to be viewed butunanticipated, would allow a viewer to review such previously-storedstereoscopic images to view the event after its occurrence. Stereoscopicimage data could be updated continuously or in some other manner to, forexample, provide a time-lapse stereoscopic recording of an event thatoccurs slowly. In embodiments where the stereoscopic image data is asingle stream, additional synchronization may be unnecessary, resultingin a simpler system, both in storing images and playing them back. Sucha system generally has the same data integrity for both the left andright views of images, as opposed to dual-data-channel systems in whicheither channel may have defects that could spoil the stereoscopic image.Additionally, such a system could also be made redundant and fail-safe,and this could be done more easily to avoid any image data degradation.Such a system could use SDs that generally have low or no recurringcosts associated with re-saving newer image data over old. The SD, orits storage media, could be removable, replaceable or expandable, withpower on or off. The SD could be of low power such that it could beincluded in a battery-operated embodiment. The SD could be of low sizeand weight such that it could be included in a handheld embodiment. TheSD could be of sufficient robustness such that it could be included inan embodiment intended for severe-environments.

Lenses

Lenses or other optical components of the camera and their mountingcould be such that interchangeable lenses could be used. Such lensescould be capable of focusing light of infrared, UV or visiblewavelengths or combinations thereof. The camera could use two lenses,one for each L/R channel, or a single lens with an optical pathswitching device such as a shuttering system and beamsplitter or acombination of these. For underwater or similar applications, a systemwherein the lenses could contact the water directly and there is a sealfor the electronic portion of the SIAD, could be used. Furthermore, theinternal volume of the sealed portion of the SIAD, possibly includingthe lenses, could be minimized and securely sealed such that noadditional outside housing would be required for use underwater,resulting in a small overall size and weight and easier use. Yet furtherthere could be a tether, thus forming a completely, possibly permanentlysealed camera head with a cable, optical fiber or wireless link to theIPU or Display. Such sealing could be potting compound.

The lenses could also be parabolic mirrors to focus directly on theimage sensors such as, for example, lenses useful for an IR imagingdevice with no IR glass optics. Alternatively, lenses could be focusedautomatically and/or together as described in the referenced patentapplication. In addition, the lenses could be small to be mountedclosely together to make a small inter-pupillary distance.

Convergence

In embodiments having two optical axes, these axes may be parallel, theymay converge at some point distant from the camera or they may beadjustable to converge at any point from a close distance to infinity orto diverge. To allow relative motion of the sensors in embodiments whereone or more of the sensors can be moved to move the optical axes, thesensors could be connected to the acquisition controller with flexiblecircuitry, which may be shielded, to allow relative motion whilemaintaining bandwidth and suppressing noise.

Convergence could be adjusted by use of a mechanism with an actuator todeflect either or both of the sensors' mounting and their respectivelens and axis. Alternatively the mechanism could move an opticalelement, for example a wedge lens, to deflect one or both axes. Themechanism could be comprised of a hinge or a flexing beam of metal,plastic or other suitable material. The actuator could be a manual screwor motorized leadscrew or cam system, or other mechanical orelectromechanical device. The control of the actuator could be designedto be operated by the camera user while shooting. The control could be aknob or switch that could also be a handle for one hand. The controlcould also be a pistol grip device where the user squeezes or activatesa lever or wheel to control convergence adjustment. Adjustable stops ordetents in the mechanism could be incorporated to aid the user toachieve their desired convergence effect. Convergence could be automatedto the autofocus or other aspects of the camera. The IPU, for example,could be programmed to recognize some moving feature and follow it withconvergence. There could be a second flexure or mechanism to adjust theaxes to 6 e coplanar, known in the art as vertical alignment.

Embodiments of the camera or camera systems may also contain aconvergence mechanism such that the user can change the convergence asthe location is being observed and/or recorded. In such embodiments theconvergence mechanism can be constructed such that it causes acontinuous change in convergence, a smooth transition that is withoutabrupt changes in the views.

The convergence device and system described herein could also be usedwith two separate cameras and camera systems, for example, without theuse of the SIAD.

Inter-Pupillary Distance (IPD)

In some embodiments having two optical axes, it may be desirable tochange the distance between the axes as measured at the camera, known inthe art as the IPD. This could be done by use of a mechanism such as abeam that moves one sensor and its respective lens and housing away fromthe other sensor, generally perpendicular to the optical axes, thatmovement taking place in the plane of the axes. A clamp could be used tomanually secure the sensor housing along a beam in the desired locationto achieve the desired IPD. Such a beam could also be used to adjustconvergence by flexing or deflecting. Alternatively the IPD could beadjusted by use of a mechanism with an actuator to move either or bothof the sensors and their respective lens and axis. Such an embodimentcould be designed to accommodate interchangeable lenses includingwithout limitation wide angle or telephoto lenses for either or bothoptical channels.

Zoom

Zoom is known in the art as changing the magnification of an opticalchannel while a location is being observed. Zooming “in” is generallyincreasing the magnification and zooming “out” is generally decreasingthe magnification. Zooming can be performed manually through the use ofa mechanism or automatically by sending an electronic signal to anelectromechanical actuator. Either technique causes, for example,changes in the axial spacing between or shape of optical elements.Generally it is desirable to have zooming performed such that abruptchanges in views do not occur.

The mechanism could be geared to drive rotational cam devices thatappropriately change the spacing between optical elements, and hence themagnification, in each optical path simultaneously. The mechanism couldbe mechanical or electromechanical.

Digital zoom is an electronic method to present a similar effect ofincreasing magnification to a viewer of an electronic display. It istypically implemented by causing a subset of the pixels within the datarepresenting a view to be expanded in order to represent the entire dataset representing the view. In this approach, additional pixels arefabricated from adjacent pixel data and interspersed between theoriginal pixels to form a view having generally the same number ofpixels as the original view. Alternatively, digital zoom can beimplemented by choosing a subset of the pixel cells illuminated by thechannel optics on a sensor, to form the data representing the view fromthe sensor of the channel.

The invention may also contain electronic mechanisms to cause digitalzoom to be applied to two channels simultaneously. Alternatively,zooming could be done on a master-slave basis whereby the master view's(the right view in this embodiment) magnification is set as desired, andthe slave view's (the left view in this embodiment) zoom isautomatically controlled to match the master. Such control could use theimage data to calculate the proper control parameters, for example bymeasuring the number of pixels between certain features common in eachL/R view and attempting to equalize them, and could be a function of theIPU or acquisition controller.

The zoom device and system described herein could also be usedindependently with two separate cameras and camera systems, for example,without the use of the SIAD.

Controls

In some embodiments the controls and functions could enable the deviceto be operated by a single person, remotely or automatically. Control ofone or more of the system's functions could be achieved through one ormore separate user interface devices used in combination, includingwithout limitation, pushbuttons or other switching devices, a touchpadscreen (,separate or attached or a part of the 3D display), a joystick,pistol-grip control device, wheel, lever, mouse or similar devicecontrolling the system that provides user feedback on the 3D displayscreen or on another screen or output indication device. Such userinterface and feedback could be stereoscopic to enhance effectiveness.Such controls could operate remotely from the device via a wireless linkor over an interface.

Communication

Communication, including transmittal of image data, and additionalcontrol of a system of these embodiments could be performed thru anexternal interface, including without limitation a network, USB,“firewire”, cameralink or custom designed interface or port, to otherdevices or to a network.

Measurements

Capture of the stereoscopic image data could include the capability tomake measurements in three dimensions, x-y-z, z being into/out of theimage plane, using the parallax data between left/right views of anypoint common to both views, to calculate the z dimension. Furthermore,it may be desirable to calculate the location of points with respect toa coordinate system. An embodiment could include a component fordetermining the location of the stereoscopic camera with respect to thedesired coordinate system, for example a GPS receiver, whereby thecoordinate system is earth's latitude, longitude and altitude. Inaddition the embodiment could include a component for determining thedirection of the z dimension with respect to the coordinate system, forexample an electronic compass and/or an azimuth indicator. The locationof points in the stereoscopic image can then be calculated with respectto the coordinate system. Quantization error of such a system could bereduced by networking more than one system viewing the desired pointsfrom different camera locations and averaging their calculated locationsof the desired points.

An embodiment could include a pan-tilt-zoom mechanism and such mechanismcould be equipped with, for example, position indication devices suchthat the direction of the z dimension could be changed and the newdirection calculated with respect to the desired coordinate system.

Deflecting Element

FIG. 3 illustrates one embodiment of the Stereoscopic Camera and Systemcontaining a deflecting element, system components, and a viewer. Thecomponents are labeled as outlined below:

-   -   1. Object in a location being imaged;    -   2. Optical axes;    -   4. Stereoscopic Image Acquisition Device;    -   5. Optical instrument, in this embodiment is a binocular with        two telescopes;    -   6. Stereoscopic display, in this embodiment an autostereoscopic        flat panel.    -   8. Viewer, in approximate viewing position in this embodiment to        see 3D image;    -   9. Image of object in location, appearing 3D to the viewer;    -   14. Deflecting element, in this embodiment a flat mirror,        mounting not shown; and    -   15 a-15 c. Wall (15 a) between harsh environment (15 b) and        non-harsh environment (15 c); wall is shown in cutaway.

In this embodiment, a deflecting element 14 can be placed in the opticalpath 2 between the object being imaged 1 and the input optical componentof the optical instrument 5. A system with a deflecting element 14 couldbe used, as a nonlimiting example, to see around corners or to protectthe camera or camera system by exposing only the deflecting element 14to an environment unsuitable for the camera 15 b while keeping most ofthe system in a safe environment 15 c, protected from the unsafeenvironment 15 b by, for example, a suitable type of wall 15 a.Electronic mechanisms can be utilized to de-reflect the views or thestereoscopic image such that the viewer 8 observes and image 9 of thelocation as if it was not reflected by the deflecting element.Alternatively, the deflecting element can be adjusted to provide otherdesirable orientation.

The embodiments and examples set forth herein were presented in order tobest explain the present invention and its practical application and tothereby enable those of ordinary skill in the art to make and use theinvention. However, those of ordinary skill in the art will recognizethat the foregoing description and examples have been presented for thepurposes of illustration and example only. The description as set forthis not intended to be exhaustive or to limit the invention to theprecise form disclosed. Many modifications and variations are possiblein light of the teachings above without departing from the spirit andscope of the forthcoming claims. Furthermore, although the presentinvention is particularly applicable to binoculars and surveillancecameras, it is also applicable to many other types of opticalinstruments.

1. A stereoscopic camera for providing visual observation of a location,the camera comprising: a binocular instrument having at least oneoptical path, wherein the instrument is capable of acquiring a pluralityof optical views corresponding to a left view and a right view of astereoscopic image of the location; and a stereoscopic image acquisitiondevice comprising a plurality of optical sensors in the at least oneoptical path, wherein the plurality of optical sensors is effective togenerate at least one data stream corresponding to at least one imagedata for each view of the plurality of optical views from the instrumentwherein at least one first data stream corresponding to the left view issimultaneously generated with at least one second data streamcorresponding to the respective right view, the at least one first andsecond data streams are combined into a third, single data streamwherein timing among the first and second data streams is known andmaintained, and the single data stream is effective to represent thestereoscopic image of the location in real time.
 2. The stereoscopiccamera of claim 1, further comprising an electronic, mechanical orelectromechanical mechanism for adjusting a first size of at least oneobject in at least one image data corresponding to at least one view ofthe plurality of optical views, whereby the first size is made equal toa second respective size of the at least one object in at least onesecond image data corresponding to a second respective view of theplurality of optical views.
 3. The stereoscopic camera of claim 1,further comprising an electronic or mechanical or electromechanicalmechanism for adjusting a first vertical position of at least one objectin at least one image data corresponding to at least one view of theplurality of optical views, whereby the first vertical position is madeequal to a second respective vertical position of the at least oneobject in at least one second image data corresponding to a secondrespective view of the plurality of optical views, whereby spuriousvertical parallax is eliminated.
 4. The stereoscopic camera of claim 1,further comprising a housing and at least one power source and at leastone device selected from the group consisting of: at least one storagedevice receptive to the single data stream, and at least one displaymechanism receptive to display the single data stream, wherein thehousing provides integration of the camera, the at least one powersource, and the at least one device, such that the camera isfree-standing.
 5. The stereoscopic camera of claim 4, wherein the camerais handheld.
 6. The stereoscopic camera of claim 4, comprising a powersource, a display mechanism, a storage device, and a housing, whereinthe housing provides integration of the camera, the power source, thedisplay mechanism, and the storage mechanism.
 7. The stereoscopic cameraof claim 3, wherein the mechanism can be operated by a single user whileobserving the plurality of optical views corresponding to the left viewand the right view of the stereoscopic image.
 8. The stereoscopic cameraof claim 1 wherein the camera comprises at least two respective opticalpaths having at least two respective optical axes, the camera furthercomprising an adjustment mechanism for aligning the vergence of therespective optical axes of each of the at least two optical paths tocoincide at a point in the location at a desired distance which can bevaried, wherein the adjustment mechanism is connected to at least one ofthe at least two optical paths, wherein the adjustment mechanismcomprises a beam connected to and holding one or more of the first andsecond respective optical paths, wherein the beam is flexible in thedirection that aligns the vergence of the axes and is inflexible inother directions and the vergence can be aligned by flexing of the beamwithout introducing spurious parallax, wherein the controls for theadjustment mechanism can be operated by a single user and whereinadjustment can be made while observing the plurality of optical viewscorresponding to the left view and the right view of the stereoscopicimage.
 9. The stereoscopic camera of claim 8, wherein the adjustmentmechanism for aligning the vergence is at least one of a trigger, aknob, a wheel, and a button.
 10. The stereoscopic camera of claim 1,further comprising a display mechanism capable of receiving thereal-time stereoscopic image single data stream and displaying astereoscopic image of the object to a user.
 11. The stereoscopic cameraof claim 10, further comprising an image processing unit formanipulating the real-time stereoscopic image single data stream,wherein the display mechanism is capable of receiving the manipulatedreal-time image single data stream.
 12. The stereoscopic camera of claim11, wherein the image processing unit for manipulating the real-timestereoscopic image data stream is affixed to the camera.
 13. Astereoscopic camera system comprising at least one camera according toclaim
 1. 14. The stereoscopic camera of claim 1, wherein each sensor inthe plurality of optical sensors further comprises a plurality ofpixels, and image data of the at least one first and second data streamscomprises pixel data, wherein the image data corresponding to a pixel ofthe left view is simultaneously generated with the image datacorresponding to the respective pixel of the right view and the firstand second data streams of the respective left and right views arecombined into a single data stream wherein timing among the pixel datais known and maintained.
 15. The stereoscopic camera of claim 14,wherein the plurality of pixels has a resolution of at least 1280×720pixels for each view.
 16. The stereoscopic camera of claim 1, whereinthe single data stream is output to another device.
 17. The stereoscopiccamera of claim 16, wherein the respective image data corresponding tothe first and second data streams comprising the single data stream areoutput simultaneously.
 18. The stereoscopic camera of claim 1, whereinthe at least one first and second data streams are combined into thethird, single data stream prior to storage of at least one image datacorresponding to the left view or the right view of the stereoscopicimage.